Changes

• changed the resampling code to be more vectorisation friendly: using restrict as pointer aliasing is guaaranteed to not happen, simplify interpolation so that it can use fma, split into fast and safe loop to reduce checks.
• reserving memory for full_text and result segments

Results

• Tested against old code.
• 1.5 - 1.8x faster resampling
• the memory reserves reduce heap thrashing as the += will trigger fewer allocations

Note

• the amount of memory reserved is currently chosen as a "reasonable " amount. probably should check for a "better" reserving heuristic
• there is a tiny drift in output of resampling due to the nature of floats. very miniscule

Summary by CodeRabbit

• Refactor
  • Optimized audio transcription pipeline for reduced memory use and faster, more reliable processing.
  • Improved audio resampling with early-exit for matching rates, a fast path for common ratios, and a robust interpolation path for arbitrary rates, increasing stability and accuracy.

Greptile Overview

Greptile Summary

This PR optimizes the WhisperCPP backend for better performance through vectorization-friendly code and memory pre-allocation strategies.

Key improvements:

• Resampling algorithm refactored to use restrict pointers and FMA-friendly interpolation (val0 + frac * (val1 - val0))
• Added fast path for integer ratio resampling (e.g., 48kHz → 16kHz uses simple stride-based decimation)
• Split resampling into fast loop and safe boundary-checking loop to reduce conditional branches
• Memory pre-allocation for full_text, segments, and word_timings vectors to reduce heap thrashing
• Changed from push_back to emplace_back with reference to avoid unnecessary copies

Minor issues found:

• Indentation inconsistency in the integer ratio fast path (lines 566-573)
• Trailing whitespace on lines 277 and 564
• The fallback loop (lines 597-609) uses push_back after calculating exact size, which could use indexed writes for better performance

Confidence Score: 4/5

• Safe to merge with minor formatting fixes recommended
• The changes are well-tested (per PR description: 1.5-1.8x faster), mathematically sound optimizations for audio resampling and memory management. Only issues are cosmetic (indentation/whitespace) that don't affect functionality
• No files require special attention - formatting issues are minor

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant Client
    participant WhisperCppSTT
    participant VectorMemory as Vector (Pre-allocated)
    participant ResampleFn as resample_to_16khz
    
    Client->>WhisperCppSTT: transcribe_internal(audio, config)
    
    Note over WhisperCppSTT: Process audio segments
    WhisperCppSTT->>VectorMemory: full_text.reserve(n_segments * 64)
    WhisperCppSTT->>VectorMemory: segments.reserve(n_segments)
    
    alt word_timestamps enabled
        WhisperCppSTT->>VectorMemory: word_timings.reserve(n_segments * 15)
    end
    
    loop for each segment
        WhisperCppSTT->>VectorMemory: segments.emplace_back()
        Note over WhisperCppSTT: Direct construction in place
        
        alt word_timestamps
            loop for each token
                WhisperCppSTT->>VectorMemory: word_timings.emplace_back()
            end
        end
    end
    
    Client->>WhisperCppSTT: needs resampling
    WhisperCppSTT->>ResampleFn: resample_to_16khz(samples, source_rate)
    
    alt integer ratio (e.g., 48kHz -> 16kHz)
        ResampleFn->>ResampleFn: Fast path: stride-based decimation
        ResampleFn-->>WhisperCppSTT: return output
    else non-integer ratio
        ResampleFn->>VectorMemory: output.reserve(output_size)
        ResampleFn->>ResampleFn: Fast loop with restrict pointer
        Note over ResampleFn: FMA-friendly interpolation:<br/>val0 + frac * (val1 - val0)
        ResampleFn->>ResampleFn: Safe loop for remaining samples
        ResampleFn-->>WhisperCppSTT: return output
    end
    
    WhisperCppSTT-->>Client: return STTResult

_{(4/5) You can add custom instructions or style guidelines for the agent here!}